🕷️️ Job Radar • SCRAPING

upwork.com 2026-04-28 🟡

🔹 Automate data extraction from NYS DOCCS Incarcerated Lookup website
👤 Client: 🇺🇸 United States Member since 2026-04-27
💰 Price: $400
🚩 Problem: Extract records bi-weekly to maintain a current database.
📦 Existing: [NYS DOCCCS Incarcerated Lookup URL]

Specifications:

[Target] Extract all available records from the NYS DOCCS Incarcerated Lookup website.
[Method] Use Python with Requests, BeautifulSoup for scraping; handle pagination by alphabetically searching last names and iterating through two-letter combinations if necessary.
[UI/UX] Not applicable
[Stack] Python (Requests, BeautifulSoup), Apify Actor
[Security] Ensure data is handled securely during transfer to Google Drive.
[Format] Output data in CSV or JSON format; automatically upload to provided Google Shared Drive.

Workflow:

1. Set up a Python environment with necessary libraries: Requests and BeautifulSoup.
2. Write a script to search alphabetically by last name, starting from single letters and progressing to two-letter combinations if needed.
3. Implement pagination handling within the script.
4. Test the script on a sample dataset to ensure accuracy and efficiency.
5. Deploy the script as an Apify Actor for automated scheduling and execution.
6. Configure automatic error alerting via Apify (e.g., email notifications if a scheduled run fails).
7. Run the script bi-weekly, ensuring successful data extraction and upload to Google Shared Drive.